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ABSTRACT 

This report presents the evaluation of the Worcester 
Operation Reading Base (ORB) Program which operated during 1971-1972r 
funded under Title I of the 1965 Elementary Secondary Education Act. 
The ORB Program was designed to provide small group remedial reading 
instruction to students in the target area schools of Worcester. 
Evaluation of the program focused on the assessment of program 
objectives^ including student reading achievement^ attitude toward 
school and self as learner^ and socio-emotional behavior. In 
addition^ the operation and implementation of the program^ as 
specified in the project proposal^ were assessed. The ORB Program 
provided small group instruction for students requiring remedial 
assistance in reading. Students attended daily classes^ 30 to 45 
minutes in lengthy conducted by a compensatory reading teacher. The 
teachers provided a variety of instructional situations designed to 
build students' basic reading skill? and to reinforce the instruction 
of the regular class. The program operated at 21 schools; depending 
on the number of eligible students^ between one and three teachers 
were assigned to a school. Participants were educationally 
disadvantaged students in Grades One through Six of Title I 
designated target area schools. Students were identified as 
educationally disadvantaged by achievement tests and by the 
evaluations of classroom teachers and school principals. 
(Authcr/JM) 
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INTRODUCTION 



This report presents the evaluation of the Worcester Title I Operation Reading 
Base (ORB) Program which operated during 1971-1972. Heuristics, Inc. was en- 
gaged in the Fall of 1971 to perform the evaluation of this program; therefore, both 
formative and summative assessment activities were conducted. The ORB Program 
was designed to provide small group remedial reading instruction to students in 
the target area schools of Worcester. Evaluation of the program focused on the 
assessment of program objectives , including student reading achievement, attitude 
toward school and self as learner, and socio-emotional behavior. In addition, the 
operation and implementation of the program, as specified in the project proposal, 
were assessed . 



DESCRIPTION OF PROGRAM 



The Worcester Operation Reading Base Program provided small group in- 
struction for students requiring remedial assistance in reading. Students attended 
daily classes, 30 to 45 minutes in length, conducted by a compensatory reading 
teacher. The teachers provided a variety of instructional situations designed to 
build students^ basic reading skills and to reinforce the instruction of the regxilar 
class . 

The program operated at 21 schools- depending on the number of eligible 
students , between one and three teachers were assigned to a school. Twenty-nine 
teachers staffed the program; each- teacher served approximately thirty students, 
four to six at a time . Table 1 presents the list of schools and the number of compen- 
satory teachers assigned to each school. The table also reports the number of 
students involved and grade levels of students served by each teacher. Many of 
the program staff members were first year teachers. Some had special certification 
in reading; the remaining teachers were certified to teach at the elementary level. 
A Program Coordinator was responsible for the overall operation of the program, 
the distribution of materials and organization of in-service training for the staff. 

In compliance with Title J regulations requiring comparable city supported 
services for target and non-target schools, a plan was formulated that would provide 
like services, but not create the problem of two remedial teachers for the partici- 
pants in ORB. The Reading Resource Teachers (city funded) did not provide 
direct instructional service to the ORB students. They did serve in an advisory 
capacity, assisting Title I reading teachers in diagnosis and continuous planning 
of instructional activities . Given the limited teaching experience of the majority 
of ORB teachers this resource and assistance role is seen as a significant factor 
in the success experienced by the ORB Program, and judged to be a sensible way 
of insuring that Title I monies supplement, not supplant, local effort. 



Table 1 



Schools in Program, Compensatory Teachers 
Assigned to Each, Students and Grade Levels 

^1 

School Number of Approximate Grades Served 

Teachers Number of 
Students 



Belmont Street Community 


3 


90 


1-6 


Cambridge Street 


1 


30 


1-6 


Canterbury Street 


2 


60 

• 


1-3 


Chandler Street 


2 


60 


1-G 


Clark Street 


1 


30 


1-3 


Edgewforth Street 


1 


30 


2-6 


Elm Park Community 


2 


60 


2-6 


Grafton Street 


2 


60 


2-6 


Harlow Street 


1 


30 ' 


1-6 


Holy Name of Jesus 


1 


30 


4-6 


Lamartine Street 


1 


30 


1-6 


Millbury Street 


1 


30 


1-6 


Oxford Street 


1 


30 


2-6 


Sacred Heart Central 


1 


30 


4-6 


Sacred Heart School 


1 


30 


2-5 


St. John ~ Ascension | 


1 


30 


1-4,6 


St. Nicholas Avenue 


2 


60 


1-6 



Table 1 (Cont . ) 



School 


Number of 
Teachers 


Approximate 
Number of 
Students 


Grades Served 


St. Stephen's 


1 


30 


1-6 


Union Hill 


1 


30 


1-3 


Winslow Street 


1 


30 


1-3 


Woodland Community 


2 


60 


1-4,6 


Total 


29 


870 





A fifteen-week in-service program was held during the Spring of 1972. 
Although participation v/as voluntary, almost all ORB teachers attended. The pro- 
gram was organized by the reading department of the Worcester Public Schools 
and included sessions on such topics as diagnosis of reading problems, phonics 
instruction, available reading materials. 



\ 

PARTICIPANTS 



Participants were educationally disadvantaged students in Grades 1-6 
of Title I designated target area schools. Students were identified as education- 
ally disadvantaged by achievement tests and by the evaluations of classroom 
teachers and school principals . 

Each school principal identified 50 eligible students for each compensa- 
tory reading teacher assigned to his school . The list specified the ten students 
(of the fifty) having greatest need for the services provided by the program 
and these ten v;ere immediately designated for the program. Twenty students 
from the remaining forty were randomly designated by the evaluator . In this 
way, a "neediest" group (ten students) was identified for each teacher, as well 
as a randomly "selected" group of twenty students, whose achievement could 



then be compared to that of the "control" group composed of the 20 students remain- 
ing from the initial fifty. Every attempt was made to restrict participation to those 
30 students originally assigned to each teacher; however, some students left the 
program during the year^and a few students were added. 



OBJECTIVES AND PRIORITIES 



The following performance objectives were formulated for the ORB students: 

1. The students will show at least one month^s growth for each 
month in the program , growth to be measured by pre and 
post-testing with the Gates-McGinitie Reading Tests > 

2. The stuaents' grades in other subjects will improve as reading 
skills improve. 

3. The students^ attitude toward school will be improved when 
opportunity for success becomes greater . 

4. The opportunity provided for success in the program will 
improve school attendance and reduce antisocial behavior. 



EVALUATION DESIGN 



A variety of evaluation activities were structured to assess the degree to 
which objectives were accomplished and to assess the operation of the ORB Pro- 
gram. These activities included: 

1 . Pre-post administration of the Gates-McGinltie Reading Tests 
to ORB students and to a group of control students; analysis 
of results to assess reading achievement. 

2. Comparlsion of 1970-1971 and 1971-1972 final grades of ORB 
students in reading, mathematics, and other school subjects. 

3 . Analysis of the Teacher Rating of Student Classroom Behavior , 
administered pre and post to ORB students, to assess their 
attitudes toward school and toward self as learner. 



4. Comparision of 1970-1971 and 1971-1972 attendance data for 
ORB students. 

5 . Analysis of the Pupil Behavior Inventory administered pre and 
post to ORB students, to assess their anti-social behavior, 

6. Construction, administration, and analysis of the two question- 
naires (December, 1971 and May, 1972) for compensatory 
reading teachers. 

7 < Construction , administration , and analysis of post-program 
questionnaires for parents, regular classroom teachers and 
school principals of program students. 

8. On-site observation ofi)rogram classes. 

9. Informal interviews with program staff members. 

10. Provision of verbal and written feedback to the project 
cocdinator . 

This report summarizes the formative evaluation activities and findings , 
then analyzes the cognitive and affective achievement data and assesses the degree 
of accomplishment of objectives . A discussion is included of the nature and fre- 
quency of program activities, as determined. by classroom observations and inter- 
views with the program teachers. The discussion focuses on the attitudes of ORB 
teachers, parents, classroom teachers, and school principles toward the ORB 
program. Finally, the report concludes with a list of commendations and recom- 
mendations for the program, based on the evaluative data collected. 



FORMATIVE EVALUATION ACTIVITIES AND FINDINGS 



Formative assessment of the ORB program was based on regular on-site obser- 
vations of program classes , discussions with program teachers , and analysis of 
responses to a questionnaire administered in December, 1971. The willingness of 
the project staff, and especially the project coordinator, to respond to recommenda- 
tions offered by the evaluators and to continually modify and improve the program 
is noteworthy and highly commendable. The ORB Program developed from a loose 
collection of reading teachers in 21 schools, to a smoothly operating, cohesive in- 
structional effort. The evalu?.tors commend the ORB staff on this accomplishment. 



/ 



The following listing presents specific problems identified by Heuristics 
and program staff during the formative assessment, and describes corrective 
action implemented by the staff in the problem areas . 



1 . The delayed arrival ojf materials for the program was related to 
late appointment of a program coordinator. Immediately after 
her appointment, materials were ordered and distributed. 

2. The lack of clear guidelines , policies , and expections created 
confusion at the beginning of the program; once the program 
coordinator finished ordering materials, her communications 
with the teachers began to clarify the program guidelines. 

3. The evaluators noted that, although the pre-serv^':.e provided 
some relevant information for the staff, it did uot completely 
satisfy their needs and requirements; thp need for in-service 
training was identified. Such in-service training was offered 
for fifteen weeks during the second semester of the school year, 
and generally seemed to meet 'ne immediate needs of the teachers. 

4. Heuristics recommended that every attempt be made to improve 
''the classroom facilities of ORB teachers. The evaluators note that 

limited action has been takery upon this recommendation , but 
recognize the inability of the program staff to take action because 
of limited availability of adequate facilities in program schools. 

5. The evaluators noted the need for increa:oed communication 
between the compensatory teachers and resource reading 
teachers. To some extent, increased communication has 
resulted from the in-service program during the second 
semester of the school year. Increased communication still 
seems to be necessary. 



7 



STUDENT ACHIEVEMENT: COGNITIVE AND AFFECTIVE OBJECTIVES 

Cognitive Domain 

Evaluation of program objectives involved the administration of standardized 
tests to assess student cognitive achievement, comparison of 1970-1971 and 1971- 
1972 final grades, and interim and final ratings on a list of behavioral objectives, 
of students in Grades 1-3. This section of the report presents a description and 
analysis of these, and a discussion of the degree of accomplishment of related 
program objectives. 

Analysis of Gates-McGinitie Reading Tests Results 

A Title I program can be considered successful only if students gained more 
than they would have gained had they not participated in the program; in this case 
the judgment is made by comparing ORB student performance to that of non- 
participating, or control group students. Thus, in addition to determining ORB 
student accomplishment of the program's cognitive objectives (Numbers 1 and 2) , 
an evaluation design was formulated which would allow comparison of the reading 
achievement of program and non-program students by a statistical analysis of their 
scores on a standardized reading test. 

As described iiT the "Participants" section of this report the selection pro- 
cedure resulted in the formation of three groups: neediest, selected, and control. 

Each of the 50 students referred for each ORB teacher was administered the 
Gates-McGinitie Reading Tests . Form 2 of the test was administered in October , 
1971 as a pre-test and Form 1 in May, 1972 as a post-test; each student completed 
the appropriate level of the test: Grade 1— Primary A, Grade 2 — Primary B , 
Grade 3~-Primary C.vand Grades 4,5, 6— Survey D. This test measures student 
reading skills in vocabulary and comprehension. The Vocabulary subtest, "samples 
the child^s ability to recognize or analyze isolated words." For each item in the 
Vocabulary subtest, the student circles the word which corresponds to the stimulus 
picture (Primary A, Primary B, and Primary C) , or circles one of four words which 
is a synonym for the stimulus word (Primary B , Primary C, and Survey D) . The 
Comprehension subtest measures a student's "ability to read and understand whole 
sentences and paragraphs." For the Comprehension subtest, the child either marks 
the picture which best illustrates the meaning of a written passage (Primary A 
and Primary B) or answers questions about a written passage by selecting one 
of four responses (Primary B , Primary C, and Survey D) . 

Each student earned a raw score (the number of questions he answered cor- 
rectly) for each subtest; raw scores on alternate forms are equivalent and, thus, 
can be statistically compared . Raw scores were transformed into grade equivalent 




scores , Reliability information on the Gates-McGinit ie Reading Tests indicates 
that alternate form reliabilities for the four levels range from . 81 to .89, and 
split-half reliabilities range from . 88 to .96. These reliabilities are considered 
satisfactory for a reading achievement test. 

This analysis assesses achievement of Objective 1, compares the cognitive 
achievement of selected and control groups, using analysis of covariance, then am- 
plifies the discussion of the analysis of covariance . 

Achievement of Objective 1 . In order to assess Objective l--"the students 
involved in the program will show at least one year's growth from pre to post- 
test on the Gates-McGinitie Reading Test the pre and post mean grade equiva- 
lent scores on both the Vocabulary and Comprehension subtests were compared, 
by grade, for all program students and for neediest students only. Table 2 reports 
the number of months gain in mean grade equivalent scores. In order to Interpret 
these results and to determine the degree of accomplishment of Objective 1 , the 
number of months reported in Table 2 should be compared to a criterion of seven 
months, the number which elapsed between pre- and post-testing. No reporting 
for Grade 1 is made because no transformations from raw score to grade equiva- 
lent existed for the extremely low pre-test raw scores of the First Graders. 



Table 2 

Months Gain in Mean Grade Equivalent Scores 



Grade 


All Prog-ram Students 


Neediest Group Only 






Months 


Gained 






Vn jabulary 


Comprehension 


Vocabulary 


Comprehension 


1 










2 


8 


6 


8 


5 


3 


9 


9 


8 


9 


4 


10 


10 


9 


9 


5 


10 


11 


9 


10 


6 


10 


12 


8 


14 



9 



A review of Table 2 indicates that all groups (except Grade 1 which was not 
included) gained more than seven months in mean grade equivalent scores on the 
Vocabulary subtest. All but the second grade gained at least seven months on 
the Comprehension subtest. Evaluators attribute this limited gain to the difficulty 
of the test. The published norms do not provide grade equivalents corresponding 
to the very low scores found in Grade 2. Therefore, the pre-test grade equivalents 
assigned do not accurately describe the achievement level on the pre-test. 
Accordingly, gains measured from that point cannot be considered a true measure 
of gain. In spite of the inability of Grade 2 students to gain seven months in com- 
prehension skills, the evaluators feel that Objective 1 was accomplished. Especially 
noteworthy are the exceptional gains of all program students in Grades 4,5, and 
6, ranging from ten to twelve montns during the seven-month period. 

Tables 3 and 4 provide the data which were used to formulate Table 2 . Also 
reported in these tables are t-tests for correlated data used to compare pre and 
post mean Vocabulary and Comprehension scores by grade for the program students 
and for the neediest students only. In every case, a statistically significant dif- 
ference at the . 001 level was indicated by the t-test for correlated data. Al- 
though some of the gain can be attributed to maturation, a consideration of 
Tables 2,3, and 4 together suggests that these statistical gains have practical 
and educational significance as well, since the students gained at least one 
month for each month in the program . In terms of absolute gains , then , the ORB 
Program seems to have been highly successful, both for the entire program group 
and for the neediest students. Figure 1 presents, in graphic form, the performance 
of students in Grades 2-6, on pre- and post-tests of each subtest. A review of 
these scores indicates that for all grades the post-test Vocabulary subtest mean 
grade equivalent is greater than the Comprehension subtest mean grade equivalent. 
This is a predictable outcome since the pre-test Vocabulary subtest mean was greater 
than the Comprehension subtest mean; also acquisition of vocabulary skills 
generally precedes acquisition of Comprehension skills by students requiring 
remediation . 

Comparison of Selected and Control Groups . Since the ultimate success 
of the program can only be measured by comparing performance of program stu- 
dents to that of a comparable group of non-program students, an analysis of covar- 
iance was used to compare the selected and control groups . (A description of the 
selection procedures is in the section titled "Participants.") Tables 5-7 present 
the results of this analysis. Tabl- 5 reports the Vocabulary subtest scores by 
grade for the selected and control groups. A comparison of the adjusted post-test 
mean raw scores of the selected and control groups indicates that, with the 
exception of Grade 2 , the adjusted post-test mean raw score of the selected stu- 
dents was higher than that of the control students. Similar results are shown in 
the analysis of the Comprehension subtest scores, as reported in Table 6. 

In spite of the fact that the adjusted post-test mean scores of the selected 
group were, with the exception of a single grade, higher than those of the control 
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group, the F-ratios obtained from the analyses of covariance were not statistically 
significant in all cases Table 7 reports that, according to the analysis of covar- 
iance, the fourth, fifth , and sixth grade selected students performed better than 
the control students at a statistically significant level on the Vocabulary subtest; 
the fourth and fifth grade selected students performed better than the control stu- 
dents on the Comprehension subtest. It is important to note at this point that 
recent evidence (Donald T. Campbell and Albert Erlebacher, in "How Regression 
Artifacts in Quasi-Experimental Evaluation Can Mistakenly Make Compensatory Edu- 
cation Look Harmful") shows that the analysis of covariance does not mathematically 
adjust to a sufficiently high level, the post-test mean score for the group which had 
the lower pre-test mean score. This limitation riiay be preventing statistically sig- 
nificant gains from being identified for Grades 1-3 in both skill areas, and for 
Grade 6 in Comprehension because the selected group^s post-test mean scores would 
not be adjusted sufficiently high to allow a statistically significant F-ratio. 

Because of these statistical limitations, the evaluators have performed a 
series of statistical analyses to determine if other evidence supports the difference 
between selected and control groups suggested by the data in Table 5 and 6 but 
only partially confirmed by the F-ratios in Tab\e 7. 

Discussion of Analyses of Covariance , One consideration was that the con- 
trol group included some students who were receiving help from the Reading 
Resource Teachers and, thereforie, should not be included in a "no help" control 
group. The control groups in Grades 1,2, and 3 included significant numbers of 
these students. (In Grade 1, control students receiving help and not receiving 
help were equal in number; for Grades 2 and 3, twice as many control students 
received extra help as did not; for Grades 4,5, and 6, the number of control 
students receiving additional instruction was negligible.) Therefore, an analysis 
of covariance between the scores of selected students and the scores of control 
students who did not receive instruction from the Reading Resource teachers , 
in Grades 1-3, was performed and is reported in Table 8. The results suggest 
that inclusion of control students receiving remedial instruction from the Reading 
Resource teachers affect only the scores on the Vocabulary subtest; therefore, 
the composition of the control group seems to have a limited relationship to the sig- 
nificance of the difference between selected and control student performance. 

Additional evidence of the impact of the ORB Program on its participants is 
suggested by the comparison of gains in mean grade equivalent scores from pre- 
to post-tests for the selected and control groups. A review of Tables 9 and 10, 
which present these analyses, suggests that for Grades 3,4,5, and 6, the selected 
group gained more than the control group in both skill areas. (As noted above, 
no transformation of the Grade 1 mean raw scores to grade equivalent scores was 
made.) Tables 9 and 10 show that the adjusted post-test mean grade equivalent 
for selected students in Grades 3-6 was greater than that for the control students in 
these grades. These results confirm the impact of the ORB Program on the students 
in Grades 4 to 6 , and provide further evidence that the program had an impact on 
the Grade 3 students. 
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Analyses which offer additional verification of the ♦ ffectivenc^ss of the ORH 
Program are summarized in Tables 11-13. T-tests for uncorrelatod <iMa wcrr 
used to compare the pre~test scores of the selected and control groups and also the 
post-test scores of the two groups on the Vocabulary and Comprehenson subtests 
(Tables 11 and 12) . 

Table 11 shows that a statistically significant difference existed between the 
pre-test mean scores on the Vocabulary subtest of the selected and control groups 
in Grades 1,3,4, and 6, with the control group performing significantly bettor 
than the selected group. A review of the results of the t tests for uncorrelat^:d 
data on the post- test scores shows that no statistically signifi?ant difference existed 
between the scores of the selected and control groups at any . ;rade level. These 
data contribute to the conclusion that the program students dio improve their 
reading skills more than the control group. Similarly, as shown in Table 12. 
there was a statistically significant difference between the pre-test scores of the 
selected and control students, in Grades 3, 4» and 6, on the Comprehension subtest. 
As m the case of the Vocabulary subtest, there was no statistically significant dif 
ference between the post-test scores of these two groups at any grade level on 
the Comprehension subtest. These results also support the conclusion that the 
ORB Program was highly effective in improving student reading achievement. 
Table 13 summarizes the data presented in Tables 11 and 12. 

A consideration of all analyses of the data from standardized achievement 
tests suggests that the ORB Program v;as very effective in improving the student 
achievement in reading. 

Analy sis of Final Grades 

In order to evaluate Objective 2--"The students' grades in other subjects 
will improve as reading skills improve"— the final grades of ORB students for 
1971- 1972 were compared to their 1970-1971 final grades by a sign test for correla- 
ted data. It is important to note that in some of the schools the traditional A to E 
grades are not assigned; instead, numerical ratings of 1 , 2, or 3 are used In 
order to analyze all data together, the evaluators converted these numerical 
ratings to the letter grades of A, C, and E, respectively. The evaluators note 
that these conversions are artificial and for the purpose of data analysis only 
Because of the increasing trend to de-emphasize final grades, and to emphasise 
a continuous non- graded approach to education. Objective 2 should be revised 
so that its wording is congruent with this new assessment philosophy. 

A summary of the analysis of the ^Inal grades is presented in Table 14. 
Tables A-F in the Appendix present the sign test analyses for Grades 1 to 6. 
Table 14 reports for each grade whether a statistically significant difference 
existed b^^tween 1970-1971 and 1971-1972 final grades for each appropriate sub 
ject area. (A hyphen denotes a subject area in which students were not assigned 
grades.) A review of Table 14 indicates a pattern of improvement in Reading, 
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Table 13 



Summary of t-tests for Uncorrelated Data Used 
to Compare Pre- and Post-Test Raw Scores of 
Selected and Control Groups on the 
Gates-McGinitie Reading Tests 



Grade 


Vocabulary 




Comprehension 




Pre Post 


Pre 


Post 


1 


Yes* 


No 


No 


No 


2 


No 


No 


No 


No 


3 


Yes* 


No 


Yes* 


No 


4 


Yes* 


No 


Yes* 


No 


5 


No 


No 


No 


No 


6 


Yes* 


No 


Yes* 


No 



* E< C 

* Significant difference favoring the control group 
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Table 14 

Summary of Statistica]ly Significant Positive Change 
in Program Students' Final Grades From 
1970-1971 to 1971-1972 



Was Change Statistically Significant? 

Grade 

Reading Arithmetic Language Spelling Social Conduct Effort 

Studies 



1 


Yes 


Yes 


No 






No 


No 


2 


Yes 


Yes 


Yes 


No 


No 


No 


No 


3 


Yes 


No 


Yes 


Yes 


Yes 


No 


No 


4 


Yes 


Yes 


No 


No 


No 


No 


No 


5 


No 


Yes 


Yes 


No 


Yes 


No 


No 


6 


No 


No 


No 


No 


No 


No 


No 
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Arithmetic, and Language in Grades 1-5 The ORB Program seemed to have the 
least impact on improving final grades o'' Sixth Grade students , and no statistically 
significant improvement occurred in any subject. Similarly, at all grade levels, 
no change occurred in the final grades for conduct or effort. These data suggest 
that while Objective 2 was only partially accomplished, the impact was of con- 
siderable practical importance because students improved their final grades in 
the fundamental skill areas of reading, arithmetic and language. 

Analysis of Ratings on Behavioral Objectives Checklists 

Interim testing in the ORB Program involved each First, Second, and Third 
Gi^ade teacher rating a sample of eight students in her class on a series of behav- 
ioral objectives which curriculum committees had identified as appropriate goals 
of instruction at each grade lev'»L The percentage of students receiving each 
rating for each objective was reported to the Project Coordinator in April , 1972 , 
and is included along with final ratings in Tables J-N in the Appendix. In order 
to determine if any gain in reading skills occurred from the time of the interim 
testing to the end of the program, each teacher rated the same eight students on the 
behavioral objectives for their grade in June, 1972. The performance of First and 
Third Grade students on an objective was assessed in one of two ways , determined 
by the objective. 

1. The students were rated as to whether they could or 
could not perform the objective, or 

2 . The attainment of the objective was rated on a scale of 
1-5 in relation to the attainment of the objective by 
other students in the class (1 represents poor perform- 
ance, and 5 describes the best performance of the 
objective in the class; 2,3, and 4 define points in the 
middle of the continuum) . 

Each Grade 2 student received a rating of 1-5 for their performance of each objec- 
tive. 

For each objective at each grade level , a sign test for correlated data was 
used to determine whether a statistically significant number of students improved 
their ratings from the interim to the final testing. The sign test analyses by grade 
level and by objective are presented in the Appendix in Tables G-I. Table 15 pre- 
sents for each grade level the percentage of objectives for which the sign test 
was statistically significant at a minimum of the .05 level. These data suggest 
that between interim and final testings, students acquired additional reading skills. 
Special increases were noted for the Grade 2 students , as 65% of the second grade 
objectives had a statistically significant z. The evaluators recommend that every 
attempt be made to use these objectives as the focus of program instruction and 
assessment during the entire program year. They should be reviewed for appro- 
priateness and completeness by the program teachers and coordinator and a revised 
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list formulated including objectives for Grades 4-6. For use in reviewing the appro- 
priateness of each objective , Tables J-N in the Appendix present , by objective, the 
percentage of students who received each rating at the time of the interim testing 
and at the time of the final testing. 

Table 1.5 

Percentage of Behavioral Objectives With Statistically 
Significant Increase in the Number of Students 
Demonstrating Mastery of the Behavior Sign Test 



Grade 


Total Number 


Percentage of Objectives 




of Objectives 


With Significant z 


1 


25 


32 


2 


20 


65 


3 


25 


50 
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student Achievement: Affective Domain 



The ORB Program also attempted to improve student attitude toward school 
and self as learner , to increase student attendance , and to reduce anti- social 
behavior. Two objectives were formulated for this domain and were assessed 
through the completion of attitude scales and a review of attendance data. 

Analysis of Teacher Rating of Student Classroom Behavior 

The ihird objective, "The students' attitude toward school will be improved 
when opportunity for success becomes greater," was evaluated by analyzing Teacher 
Rating of Student Classroom Behavior , completed by regular classroom teachers, pre 
and post, in November, 1971 and May, 1972, respectively. This twelve-item scale 
is composed of two subscales: 1) Attitude Toward School and 2) Attitude Toward 
Self as Learner. For each child the teacher was requested to indicate whether he 
"strongly agreed," "agreed,*^ "disagreed," or '^strongly disagreed" with each of 
the twelve statements on the scale. A rating of +2 to -2 was assigned to each re- 
sponse, according to whether it reflected a positive or a negative attitude of the 
student. These scores were then summed for the items on each of the subtests. 
The score on the five-item Attitude Toward School subscale, which included such 
statements as "He likes school" and "He thinks that school is uninteresting," could 
vary from -10 to +10, reflecting a negative to positive attitude toward school, 
respectively. The Attitude Toward Self as Learner subscale, with seven items, 
including "He often gets discouraged in class" and "He works up to his ability," 
could range from -14 to +14, reflecting a highly negative to highly positive atti- 
tude toward self as learner. 

Table 16, which summarizes the t-tests for correlated data used to compare 
pre- and post-ratings by grade level on the Attitude Toward School scale reports 
a statistically significant change in attitude for Grades 1 and 2 only. Grade 1 stu- 
dents showed positive change in attitude whereas Grade 2 students showed negative 
change. Because of the lack of a consistent direction in the change of student atti- 
tude toward school, it is difficult to explain the results or to relate them to parti- 
cular processes in the classroom. It is important to note that in all cases both pre 
and post attitudes toward school were positive, making positive change unlikely. 

Table 17, which reflects changes in the students' attitudes toward themselves 
as learners, showed a similar lack of consistency in the difference between pre 
and post ratings at different grade levels. Only the Grade 3 and 5 students demon* 
strated a statistically significant difference between pre and post attitude toward 
self as learner. The Third Graders showed a positive change, whereas Fifth 
Graders showed a negative change. Again, it is difficult to explain these results 
in terms of the particular processes used in the ORB Program . The evaluators 
recommend, however, that if improvement of attitude toward school and attitude 
toward self as learner remain objectives of the program, specific steps should be 
taken to accomplish them. 
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The third program objective was accomplished to a very limited degree as 
only one grade demonstrated improvement in attitude toward school, and another, 
improvement in attitude toward self as learner. 

Analysis of Student Attendance Data 

The fourth objective of the ORB Program focused on improving student atten- 
dance and reducing participants' anti-social behavior. The project^s rationale 
was that provision of extra attention and success experiences for the student would 
result in improving their attitude toward school , and, consequently, their attendance. 

A t-test for correlated data was used to compare the number of days pro- 
gram students were absent from school, during the 1971-1972 school year-- 
the year of ORB Program participation—to the number of absences during the 
1970-1971 school year. Table 18 summarizes these tests of significance and re- 
ports the mean number of days students were absent during the two years. A 
review of Table 18 indicates that for all program students, a statistically signi- 
ficant difference favoring the 1971-1972 school year existed. This analysis sug- 
gests that at least part of objective four was accomplished. A more detailed analy- 
sis of attendance, by grade (Table 18) , shows a statistically significant differ- 
ence between attendance in 1971-1972 for students in Grades 2,3, and 6. Their 
attendance improved during the program year, with the mean number of days 
absent being less in 1971-1972 than in 1970-1971. 

Analysis of the Pupil Behavior Inventory Scores 

As part of the measurement of the fourth objective , the Pupil Behavior 
Inventory was also completed as a pre-test and post-test, in November, 1971, 
and May, 1972, by each ORB student's regular classroom teacher. This scale 
is composed of four subscales: Classroom Conduct, Academic Motivation and 
Performance, Socio-Emotional State, and Teacher Dependence. 

For each item on the Inventory the teacher indicates whether the student 
demonstrates the behavior 'Very frequently," "frequently,'' "sometimes," "in- 
frequently," or "very infrequently." These ratings are transformed into a numer- 
ical scale ranging from 5 to 1 , with 5 representing either a very frequent demon- 
stration of a positive behavior, such as "shows initiative," or a very infrequent 
demonstration of a negative behavior, such as "disrupts classroom procedures." 
The ratings for all items in a dimension are summed; this total is then divided by 
the number of items in the dimension, yielding a dimension score between 5 and 
1, with 5 representing extremely desirable behavior, and 1 representing extremely 
undesirable behavior. 
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Table 18 

A Comparison of 1971-1972 
Program Student Attendance 



Grade 


Days Absent 
1970-1971 




Days 
1971 


Absent 

-1 Q79 








X 


SD 


X 


SD 




t 


1 (N=51) 


24 


.75 


16.69 


18, 


.61 


12.61 


1. 


.781 NS 


2 (N=123) 


20 


.85 


15.87 


15, 


.99 


14.99 


3. 


.790* 


3 (N=131) 


18 


.79 


17.79 


12. 


.37 


13.18 


5. 


.389* 


4 (N=123) 


13 


.228 


12.834 


12 


.407 


13.429 


0, 


.623 NS 


5 (N=106) 


14 


.95 


15.61 


13 


.91 


11.99 


0, 


.738 NS 


5 (N=91) 


13 


.91 


12.59 


10 


.76 


12.66 


2, 


.716* 


Total 
(N= ) 


17 


.25 


15. 79 


13 


.64 


13.81 


6, 


.625* 



NS - Not Significant 
* p < .001 



Tests for reliability and validity indicated that the Pupil Behavior In- 
ventory meets the criteria for a reliable and valid measurement instrument. Re- 
liability was determined, using a modified split-half analysis, by d. ^ding in half 
randomly both a normal group and a malperforming group of students and then 
using a t-test to compare , within each group , the performance of the two halves. 
Only two of the thirty-two comparisons reflected statistically significant differences, 
confirming the high reliability of the Inventory. The validity of the test was deter- 
mined by comparing the performance of a group of students identified as normal 
and another identified as malperforming on this test. The statistically significant 
difference between the performance of the two groups according to their ratings on 
the Inventory reflects its validity. 
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To determine the effect of program participation on anti-social behavior in 
students , a t-test for correlated data was used to compare pre and post scores on 
the Pupil Behavior Inventory . 

The Classroom Conduct dimension is a 12-item scale which assesses the 
classroom behavior of a student by rating his behavior among other students and 
his relationship to the teacher. Two examples of items on this scale are ^'resistant 
to teacher" and "aggressive toward peers." 

The Academic Motivation and Performance dimension "focuses on the pupil's 
motivation toward and performance of academic tasks." This dimension includes 
such items as "hesitant to try, or gives up easily," and "uninterested in subject 
matter." The test manual notes, "Although ratings of pupils on this dimension 
should correlate closely with course grades, experience suggests that the FBI 
is more sensitive to teacher perceptions of changes in student motivation than 
course grades . " 

The Socio-Emotional State dimension includes five items designed to assess 
the emotional and social adjustment of the student, especially his interaction with 
his peers. Items in this dimension include "friendly and well received by other 
pupils," and "isolated, few or no friends." 

The fourth dimension assessed was Teacher Dependence, which provides a 
measure of the student's need for reassurance from the teacher. This dimension 
contains only two items: "seeks constant reassurance" and "possessive of teacher." 
The manual suggests that this dimension successfully identifies withdrawn behavior. 
The evaluators note that, in most cases where a program encourages a child to seek 
reassurance and help from a teacher, the scores on this dimension should decrease. 
In addition, one might expect that older students would receive higher scor^JS on 
this dimension than younger students who, because of age and lack of experience, 
would generally be more dependent upon the teacher. 

For each dimension a t-test for correlated data was used to compare pre- 
and post-test mean scores. A review of Table 19 indicates that only for the Socio- 
Emotional State and Teacher Dependence dimensions did a statistically significant 
difference exist between pre- and post- test scores. For the Socio-Emotional State 
dimension the post-test mean score was higher than the pre-test mean score, 
reflecting higher teacher rating of Socio-Emotional State. This improved social- 
emotional behavior suggests that the positive reinforcement aiid success experiences 
provided by the ORB teachers had a positive impact on the students. The reduction 
in rating of teacher dependence most likely reflects the students' increased ability 
to relate to the teachers, also a positive outcome. The Classroom Conduct ratings 
remained the same and the Academic Motivation and Performance mean ratings 
declined from pre- to post~test. 
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Table 19 

Comparison of Pre and Post Scores of 
Program Students on Pupil Behavior Inventory 

(N = 611) 



Dimension 




Pre- 
X 


-Test 
SD 




Post- 
X 


-Test 
SD 




t 


Classroom 
Conduct 


3 


.409 


0.747 


3 


.413 


0.978 


0 


.139 NS 


Academic 

Motivation 

and 

Performance 


3 


.512 


0.803 


2 


.888 


0.897 


1, 


.769* 


Socio- 

Emotional 

State 


3 


,512 


0.658 


3, 


.686 


0.770 


5, 


,824* 


Teacher 
Dependence 


3 


,777 


0.903 


3 


,125 


0.959 


13, 


,323* 



NS - Not Significant 
* p< .001 



Evaluators feel that one explanation for the inability of the program to reduce 
student anti-social behavior in all areas measured by the Pupil Beha'VLor Inventory 
may be the lack of specific processes focused on this goal. Obsei v ation of pro- 
gram classes suggests that, although many of the teachers provided positive rein- 
forcement and success experiences for the students, thes;; processes might have 
reduced anti-social behavior only in the ORB classroom vvithout any carryover to 
the regular classes except for the students^ socio-emotional state. The evaluators 
feel that the accomplishment of this objective depends on the introduction of 
specific activities and processes designed to reduce anti-social behavior. Such 
activities should include processes which would extend the reduction of anti- 
social behavior to the students^ regular classroom. 
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STUDENT ACHIEVEMENT: SUMMARY 



The analyses of achie vement data indicate that the ORB Program had a 
significant impact on the cognitive skill development of the students. Not only 
did program students gain ac least one month in mean grade equivalent score 
on the Vocabulary and Comprehension subtests of the Gates-McGinltie Reading 
Tests for each month of program participation , but they performed better than a 
comparable control group on this standardized achievement test. Gains were 
also reflected in final grades of students in Grades 1-5 and by the specific be- 
havioral objectives, particularly for Grade 2 students. It is therefore concluded 
that the program was highly successful in achieving the stated cognitive objec- 
tives . 

The impact of the program on the affective domain was more limited. Al~ 
though a reduction in absenteeism occurred, no aubstantive improvement in 
attitude toward school or attitude toward self as learner, or reduction in anti- 
social behavior was effected. 



riASSROOM OBSERVATIONS 



One of the major evaluation activities was on-site observation of program 
classes for the purpose of documenting the type of instructional activities and 
materials used in the program. Because a detailed discussion of these observa- 
tions was presented in the Interim Report, completed and suomitted in April, 
1972, oiJy a brief listing of the observation findings is recorded here and, 
where appropriate, related to achievement findings, (Tf the reader is interested 
in a detailed discussion of classroom observations , -accompanied by a documen- 
tation of occurrence for each class , he should review Appendix A which excerpts 
relevant sections of the Interim Evaluation Report,) 

1. Program scheduling varir^d in different classes , the most common 
scheduling was five periods of instruction daily for six students 
each, although class size frequently decreased because of student 
absences . The evaluators feel that the most satisfactory schedule 
was a rotating one in which students received instruction at dif- 
ferent times each day of the week so that they did not miss the same 
classroom activities each day during the ORB period. Sufficient 
cooperation existed between the ORB staff and the regular class- 
room teachers, so that ORB classes were scheduled at the optimum 
time for both of these groups , as well as for the ORB students. In 
some cases, however, 0R3 teachers did not receive as much co- 
operation from the school principals and classroom teachers as 
would have been desirable for maximum program effectiveness . 

2. The student selection process for the program has been described 
in this final report's section, "Participants." Formulation of a list 
of recommended students was based on different considerations in 
each school, including recommendations of principals and classroom 
teachers , and available achievement scores or other ratings of reading 
performance. Receiving greatest consideration were the recommenda- 
tions of ihe 1971-1972 classrooM teacher, who had only a brief ac- 
quaintance with students in her class before being asked to make 
recommendations fur the ORB Program. Evaluators suggested in the 
Interim Report that recommendations of ORB students be made by the 
classroom teacher of the year prior to program operation, for example, 
by the 1971-1972 classroom teachers for the 1972-1973 ORB Program. 

A final validation of eligibility ai.d formulation of a list of students 
could then be made by the current classroom teacher and school 
principal The evaluators also noted the need for more stringent and 
uniform criteria in the identification and selection of pupils for parti- 
cipation. Selection of inappropriate students luay have prevented 
even greater gains , since some students with severe emotional prob- 
lems were selected for the program. 
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3. Program effectiveness was primarily a function of teacher talent al- 
though other factors, such as classroom facilities, cooperation between 
ORB teachers and classroom teachers, and cooperation between ORB 
teachers and reading resource teachers, contributed to it. Consider- 
ing the success of the ORB Program as reflected by achievement dat;i. it 
may be concluded that teacher talent and communications were excel- 
lent. 

4. Although individualized instruction was a goal of the progi'am , the 
relatively inexperienced ORB teachers had difficulty implementing 
it. During the year increasing individualization of instruction w^as 
observed as thu teachers became more experienced and comfortc 

in the classroom and with the program. Nr.vertheless , the evaluators 
noted the need for greater individualization in the prcjram and sug- 
gest that this "^e encouraged during the 1972-1973 school year. 

5. The activities most frequently observed in ORB classes were the 
students' completing of individual reading assignments , reading of 
texts as a group, working with flashcards or word drills, completing 
worksheets, and playing games. The frequency of performance of each 
of these activities within each classroom varied. Table 20 shows , 

for each activity » the percentage of classes receiving each of four rat- 
ings (0= no occurrence, 1= some occurrence, 2= moderate occurrence, 
and 3= extensive occurrence) . This table suggests that greater 
emphasis was placed on using individual reading assignments and 
playing games, and less on reading texts as a class, using flash 
cards or word drills , and completing worksheets. The evaluators 
feel that the use of individual reading assignments reflects some 
individualization of instruction , buc that greater individualization 
might result in greater achievement go.ins by program students. The 
playing of games was popular as a means of improvirj^ student attitude 
toward reading yet the lack of positive improvement in student attitude 
toward school and self as a learner suggests that this process did not 
result in significant improvement in attitude. 

The presence of other characterisitics in the classroom^; is described in 
Table 21. The evaluators feel that using audio-'^'isual equipment, varying activi- 
ties during the instructional period, employing innovative techniques in instruction, 
providing success experiences for students , and conducting inaividualized instruc- 
tion are positive characteristics of a classroom; ideally, all classrooms should re- 
ceive ratings of 3 on these factors. A review of Table 21 indicates, however, that 
fewer than one~half of the classes were characterized by either moderate or exten- 
sive use of these instructional activities . Increasing the use of these activities 
might lead to academic gains and improved program operation. 
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Table 20 



Classroom Activities and Materials 
Rating* Percentag^es 



Category 




Rating 






0 


1 


2 


3 


Individua] Reading Assignment 


31 


17 


41 


10 


Reading- Text as Class 


62 


31 


6 


0 


Flashcards or Word Drills 


48 


41 


10 


0 


Worksheets 


17 


79 


3 


0 


Games ^ 


17 


44 


34 


3 


Table 21 










Classroom Characteristics Rating Percentages 








Category 




Rating 






0 


1 


2 


3 


Use of Audio/Visual 
Equipment r 


48 


27 


24 


3 


Varies Activities During 
Program 


10 


44 


41 


3 


Innovative Techniques 


44 


27 


24 


3 


Success Experiences 


3 


55 


41 


0 


Individualization 


17 


41 


31 


10 


Availability of Classroom 


45 


28 


24 


3 



ATTITUDES AND OPINIONS 



Data on attitudes and opinions were collected through the administration of 
questionnaires to ORB teachers, students' classroom teachers, school principals, 
parents, and through informal interviews with project personnel, conducted 
periodically during the program year. 

In December, 1971, a detailed questionnaire was distributed to all ORB 
teachers to determine their reactions to various aspects of the program. The re- 
sponses to this questionnaire were then analyzed in a written process report as 
part of the formative assessment of the program. A follow-up on problems 
identified in the section title "Formative Evaluation Activities and Findings" indi- 
cates that, in general, attitudes toward these problems changed because of cor- 
rective action implemented in the program — for example, materials were secured 
and communication within the program was increased through regular in-service 
and staff meetings . 

The questionnaire completed by 21 teachers in May, 1972 encouraged them 
to cite strengths and weaknesses of various aspects of the program and to offer 
suggestions for program modifications. The 53 classroom teachers, 15 school 
principals and 226 parents who returned post-program questionnaires reacted 
to a series of statements about various aspects of the program. A discussion of 
these reactions, by topic, follows. 



Program Implemention: Scheduling 

Different schedules were followed in different schools, according to their 
individual characteristics. ORB teachers were asked to describe the size, length, 
and frequency of their classes. The number of classes conducted by each teacher 
ranged from five to eight, with class size ranging from four to six; the largest con- 
centration of classes was in the 5-daily group, followed by six, seven, and eight 
in decreasing order of occurrence. Usual class size was six students, followed by 
five and four Most classes met for forty-five minutes daily , although four teachers 
held classes for forty minutes , three for thirty-five , and one for fifty minutes a 
day. In general, the larger the number of classes scheduled daily the smaller the 
class size and the shorter the instructional period. 

In contrast, when questioned about the optimum number of students to be 
served per period and the optimum length of instructional time, the majority of 
ORB teachers felt that four students should be served; five, six and three were 
the other responses cited in decreasing order of preference by the ORB teachers. 
An equal percentage of ORB teachers felt periods of thirty, forty , and forty-five 
minutes would be ideal. The majority of teachers (90%) saw each student daily, 
although a few teachers saw the students less frequently. One teacher reported 



teaching five percent of her students only two days a week. Other teachers re- 
ported teaching fr;om five to ten percent of their students three days a week . The 
evaluators feel that all students, unless absent from school, should attend ORB 
classes daily, since less than regular attendance may limit program impact. 
When questioned about the minimum number of times they should work with 
each student during the week, one teacher said two, seven teachers said three, 
six said four, and eight said five. All but one teacher agreed that the maximum 
number of sessions per week should be five. 

Classroom teachers were asked to suggest ways for improving the ORB 
class schedule so that it would be better suited to student needs. The most fre- 
quent response reflected satisfaction with the current schedule. Other suggestions, 
each generally made by one teacher, included admitting and dismissing students 
at various points during the school year, developing a more flexible schedule, 
having the program teachers work with the entire reading grouD , and increasing 
the relationship between the ORB lessons and the classroom lessons. The eval- 
uators concur with the last suggestion. In many cases this relationship existed, 
but was limited. The evaluators , as well as Title I guidelines , disagree with the 
suggestions of admitting and discharging students during the program year and 
of teaching the entire reading group. The evaluators feel that emphasis should 
continue to be placed on teaching a limited number of students for the entire 
school year; too often when the students are returned to the classroom they im- 
mediately begin to perform poorly. It has been shown that short term instruction 
has only a short term effect and, therefore, should not be encouraged. 

Program Implementation: Administration 

The ORB teachers and the school principals were encouraged to react to the 
administration of the program, to the role of the project coordinator and its impact 
on project operation, to the general planning and structuring of the program, to 
the role of the school principal, and to impact of the program in the school. The 
ORB teachers reported that the project coordinator had been of great assistance in 
providing materials , general help , and advice when needed. Typical responses 
indicated that she was helpful in: 

"Either answering questions or being able to direct me to where I 
could get my questions answered." 

"1) Securing supplies, 2) helping in creating my program, 
3) advice on teaching problem learners, 4) general support." 

"An all-round general way of helping me most when I need it." 

Other teachers commented that she had served as liaison among ORB teachers , 
checked on their progress in teaching and the progress of the program , and served 
as a bank for requests and complaints . The school principals , with two exceptions , 
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agreed that the project coordinator had kept them well informed about every aspect of 
ORB. Requests for additional assistance from the coordinator by some ORB teachers 
included requests for more conferences and visits, comments on their work, aid 
in diagnosing student problems, curriculum organization, and liaison between 
them and the classroom teachers. It should be noted that each of these suggestions 
was made by one to three teachers, so they do not necessarily reflect the opinion 
of the entire ORB group. The evaluators do feel that much of the project coordi- 
nator's time at the beginning of the 1971-1972 school year was devoted to ordering 
and distributing materials . establishing rapport with and encouraging acceptance 
of the program by school principals. Discussions with the coordinator indicate 
that during 1972-1973 she intends to spend more time in the ORB classrooms, 
providing more direction and supervision of program instruction. 

Reactions to the assistance provided by the school principals were equally 
positive. Eight teachers cited that the greatest assistance provided was a compos- 
ite of things that the principals had done for them . Other areas cited by teachers 
included providing work areas, making materials available, allowing the individ- 
ual teachers to implement the program as they felt was appropriate and in line with 
the guidelines, and offering support and cooperation. Six teachers said they would 
like additional assistance from the school principal, in the areas of coordination 
with the regular classroom teachers, communication with the school department, 
and discipline. That more of the ORB teachers did not ask for additional assis- 
tance suggests that they were satisfied with the support and cooperation of the 
principals. This cooperation is further evidenced by the fact that thirteen of fif- 
teen principals disagreed that the presence of ORB teachers in the building had 
created administrative problems. Similarly, twelve principals felt that the ORB 
program was well organized, planned, and administered. 



P rogram Content (Materials, Activities, Objectives) 

The effectiveness of the program frequently depends on the appropriateness 
of, and the staffs understanding of, its objectives; it is important that the profes- 
sionals involved understand what the program is attempting to accomplish and that 
these accomplishments will be appropriate for the target students. Program teachers 
were asked to state if they understood the objectives of the program in the December 
questionnaire; the January Process Report stated that a large majority of teachers 
had a full understanding of the program objectives. Other professionals involved 
in the program, such as classroom teachers and school principals, were asked, on 
the post-program questionnaires, to indicate whether they understood the objectives 
of the program. Since both groups were in almost unanimous agreement that they 
understood the objectives of the ORB Program, it is evident that program effectiveness 
was not limited by lack of understanding of objectives. Responses also suggest that 
the objectives of the program were appropriate for the ORB students; 50 of the 53 
classroom teachers who responded agreed that the objectives of the ORB Program 
met the needs of the students. Therefore, we can conclude that program effective- 
ness was not limited by inappropriate objectives. When the program teachers were 
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asked to cite which objectives the program should include and which it should 
exclude, many teachers did not cite examples. Suggestions for additional objec- 
tives , offered by individual ORE teachers , included those which would focus on 
arithmetic skills, social development, creativity, independence, and emotional 
growth. Some responses to this question reflected a negative reaction to the test- 
ing program, the focus on achievement gains, and the emphasis on test scores. 

Another aspect of a program which could limit its effectiveness is the lack 
of appropriate quantity and/or quality of materials. Therefore, the ORB teachers 
were asked to list those materials which they felt had been most valuable in 
instruction and those materials for which they still had the greatest need. The 
most valuable materials cited, in order of frequency of response, were workbooks, 
games, audio-visual equipment — especially the cassette player — sequential phonics 
masters, paperbacks, and other reading texts. The materials for which teachers 
still had the greatest need generally reilected their individual preferences. Eight 
teachers asked for interesting books , four asked for more workbooks , school sup- 
plies , audio-visual materials, and cassette taped reading lessons. Other responses 
listed commercially prepared games, art supplies, plays, poems, additional read- 
ing materials , and ditto masters . The evaluators recommend that each program 
teacher be polled immediately to determine their materials needs and every attempt 
be made to redistribute materials within the program and to order additional mater- 
ials . 

Although the number of students in each class was limited, to decrease 
student-teacher ratio and thereby increase effectiveness of instruction , another pro- 
cess goal of the program was to implement individualized instruction . As noted 
in the section, "Classroom Observations," few teachers individualized instruction 
to an extensive degree — the general pattern was to individualize only occasionally. 
On the end-of-program questionnaire the ORB teachers were requested to iden- 
tify the problems in individualizing instruction. The most frequent response is 
described by the following quotation "not having a room or place away from the 
classroom to work in." The next most frequent responses include class size, 
heterogeneity of groups, and lack of appropriate materials. The evaluators dis- 
agree that group size, heterogeneity, or lack of interesting materials should have 
limited individualization but do recognize that space arrangements create problems. 
They suggest that the project coordinator and individual teachers discuss ways in 
which the use of space can be improved so that this aspect of the program can be- 
come operational. 



Communication and Coordination 

Three types of communication were involved in the operation of the ORB 
Program: ORB teachers with classroom teachers , ORB teachers with reading 
resource teachers, and ORB teachers with each other. The evaluators noted 
that these communications were generally informal, and at times limited. ORB 
teachers and classroom teachers were asked to suggest ways for improving com- 
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munication and coordination of effort between their groups. Although several 
ORB teachers were satisfied with communication and coordination , suggestions 
for improvement included holding regular meetings, encouraging a greater under- 
standing by the classroom teachers of the compensatory teachers' responsibilities 
and eliciting their cooperation. ORB teachers offered the following comments: 

"I would recommend that, in the future, a brochure would be 
made up explaining the scope, guidelines and aims of the pro-- 
gram. I found that some of the teachers, parents, and the prin- 
cipal were not quite clear about these aspects of the ORB new 
program." 

"If possible , there should be a joint meeting of compensatory 
teachers and classroom teachers to explain our presence in their 
classes. And the guidelines of the program should be outlined, 
discussed and altered if necessary to avoid many of the problems 
we had this year ." 

The classroom teachers and school principals were also asked to react to 
the issue of communication and coordination. The classroom teachers unanimously 
agreed, with 40 of 53 strongly agreeing, that the compensatory reading teachers^ 
instruction provided valuable support for the regular classroom instruction. All 
principals perceived and reported the satisfaction of the classroom teachers. Simi- 
larly, all but three classroom teachers stated that sufficient cooperation and communi- 
cation existed between them and the compensatory teachers; this response was con- 
firmed by all school principals . The classroom teachers were also asked to make 
suggestions about how to improve communication . The majority felt that it was "very 
good now." Suggestions made by the remaining teachers for improving communication 
and coordination included increasing the length of planning sessions , scheduling 
monthly meetings to discuss progress and regrouping, and generally increasing 
communication. One teacher suggested the need for informal progress notes about 
the students , and another desired a clearer explanation of the program at its begin- 
ning. The evaluators feel that the scheduling of regular meetings or conferences 
between ORB teachers and regular classroom teachers would be a valuable asset 
to the planning of reading activities for both teachers , since the majority of classroom 
teachers (35) responded that the compensatory reading teacher generally covered 
lessons other than those covered in the regular class. 

Communication between the ORB teachers and the reading resource teachers 
was an essential component of the ORB Program. Communication with the resource 
teachers was not a problem for the majority of the ORB teachers. Ten reported 
that they were satisfied and numerous teachers made no suggestions about how to 
improve communication. Two suggestions for improvement were scheduling more 
meetings between the two teachers and working together more on reading problems 
and related teaching techniques. The evaluators feel that both these suggestions 
should be considered. 

ERIC 



Similarly only a few individuals made suggestions about ways to improve 
communication among the ORB teachers. These suggestions include more frequent 
meetings, exchanging of audio-visual supplies between and within schools, dis- 
tributing a bulletin about the program , and initiating inter-classroom visitations , 
The in-service course made a positive contribution to improving communication 
among ORB teachers , The evaluators recommend that a similar in-service pro- 
gram be held during future ORB Programs . 

Orientation and Training 

Both the ORB teachers and the classroom teachers were asked to suggest 
areas on which pre-planning and orientation should focus. Suggestions made by 
ORB teachers included such topics as screening, scheduling, discussing and 
formulating a curriculum guide for ORB classes, discussing the meaning of the 
program and its goals , distributing materials , and determining needs of students 
and planning instruction to meet those needs. The classroom teachers' sugges- 
tions included ways of increasing their rapport and interaction with compensatory 
teachers and discussion of instructional material to be used in the ORB Program, 
Some respondents also cited areas of instruction on which they felt the ORB 
teachers should focus — comprehension and phonics skills. 

The ORB teachers were also questioned about the most and least useful 
aspects of the current in-service program. Reported as the most valuable as- 
pects were, exchanging ideas with other teachers, discussing phonetic develop- 
ment and behavior modification, being exposed to different speakers and a variety 
of reading systems , and viewing new materials and their use. The list of least 
valuable aspects of in-service included the talks of certain speakers, the behavior 
modification lectures, the explanation of testing, the introduction of in-service 
late in the year, and exhibits of materials . 

The ORB teachers were further encouraged to cite in-service training 
which they thought would be of most assistance to them. Responses reflected in- 
dividual preferences of the teachers and included sessions about formulation of 
a curriculum guide , methods of overcoming reading problems and increasing 
comprehension; ways of using audio-visual materials. Other suggestions in- 
cluded sessions on sequential phonics , practical ideas for teaching phonics, 
individual programmed materials , behavior modification techniques, and teach- 
ing students with emotional, or motivational or disciplinary problems. The 
evaluators feel that each of these topics has potential for inclusion in an in- 
service course , and note that individual preferences of the teachers should be 
considered in formulating the in-service topics. 

Teacher and Parent Perception of Program Benefits and Student Reactions 

The classroom teachers and parents of program students were asked to 
describe the benefits of the program and students^ reactions to it. The teachers 



45 



and parents were in almost unanimous agreement that the students enjoyed going 
to the reading class , with 98 percent of the teachers and 94 percent of the parents 
responding positively. A similar reaction was obtained — from 85 percent of the 
parents and 96 percent of the classroom teachers — to the question of whether the 
ORB students had demonstrated significant improvement. Related comments by the 
parents included: 

"I have seen considerable improvement in all my daughter's 
marks. Before she went to the reading class she was falling 
behind the other children because of her reading problem." 

"This has helped my child a lot and I hope she can continue in 
it." 

In further describing this improvement by citing the most generalizable 
academic and social benefits to the children receiving assistance from the ORB 
Program, 21 classroom teachers cited the ability to read; 20 cited the students' 
increased confidence in themselves , 13 cited the ability to assist one another; 
six cited the individual attention the students received in weak areas; four cited 
word analysis and vocabulary development; four cited the ability to function in- 
dependently in a normal classroom. The program was also perceived as having 
impact on the children's social skills , as all but two classroom teachers felt that 
the ability of ORB atudents to function in a group and to function individually 
had improved during the year. 

Another effective gain was reflected in the reporting of 90 percent of the 
parents that as a result of attending the reading class their child seemed to enjoy 
reading more. 

Community Awareness and Reactions 

The parents were asked three questions to determine the extent of their 
awareness of the existence and content of the ORB Program. Ninety-eight per- 
cent reported that they had been aware of their children's participation in p spe- 
cial reading program during the school year. Ninety-three percent reported 
that their children had informed them of some of the activities conducted in the 
reading class. The ability of almost all parents (93%) to name their child's ORB 
teacher further reflects their knowledge of the existence of the program. The 
parents' enthusiasm for the program and its accomplishments is reflected in the 
following comments volunteered by the parents: 

"I would like to see this special reading class go on. It has 
helped my son very much. " 

"It is a very good program for the children hope they will 
continue this program for the children who need help as 
0 mine did, Thank you." 
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CONCLUSIONS , COMMENDATIONS , AND RECOMMENDATIONS 
Conclusions 

The overall evaluation of the Worcester ORB Program is that, after getting 
off to a rough start, it was importantly successful. It produced measurable gains 
in cognitive achievement, won the support of school personnel and parents, and 
justified the expenditure of the Title I funds which made it possible. 

Much of this success was attributable to good planning, good training, and 
cooperation between school and program personnel. 

Data gathered about student achievement indicate that in cognitive areas 
program students made significant gains during the seven-month instructional 
period and did so at a significantly higher rate than control group students . Even 
greater gains might have been possible had there been more individualization of 
instruction . 

Success in the affective domain was more limited, but that may have been 
because the program did not include specific affective behavioral objectives, or 
activities designed to accomplish those objectives. 

Teachers involved in evaluation of the program were in unanimous agree- 
ment that compensatory instruction provided valuable support for regular classroom 
instruction . 

The program staff should be highly commended for the impact of the ORB 
Program . 

Recommendations 

On the basis of the findings reported, the evaluators recommend that: 

1. Specific processes be directed toward encouraging positive 
affective development. 

2. Behavioral objectives be formulated, where necessary, and used 
during the entire school year. 

3. The pre-service sessions be revised and restructured to focus 
more on the particular needs of the ORB teachers. 

4. Orientation sessions be scheduled to include regular classroom 
teachers . 



Greater communication between ORB teachers and reading resource 
teachers be encouraged. 



6. Greater coordination between ORB teachers and classroom teachers 
be encouraged by structuring regular meetings or conferences. 

7. Instruction be more individualized. 

8. In-service programs be continued and addressed to the particular 
needs of program teachers, including discussions of methods for 
individualizing instruction. 

9. Every attempt be made to secure appropriate classroom facilities 
where they are currently inappropriate. 

10, Materials be ordered earlier. 

11. Teachers be polled to determine the materials they need and mater- 
ials be redistributed within the program or ordered as soon as 
possible . 

12 . Additional materials be ordered for older students . 

13. Students be selected by those teachers who were their classroom 
teachers at the end of the year prior to program operation. 

14. Increased diagnosis of needs and prescription of instruction be 
done by the ORB teachers in cooperation with the children's 
regular classroom teacher . 

15 . The Gates-McGinitie Reading Tests be replaced by the Metropolitan 
Reading Tests , 1970 edition. 

Commendations 

The program has earned specific commendations for the following: 

1. The reading achievement gains of the students. 

2. The improved final grades, in other subject arev , of students 
on some grade levels . 

3. The decreased absenteeism during 1971-1972 as compared to that 
in 1970-1971. 

4. The interest, enthusiasm, and high caliber of project staff. 

5. The increasingly smooth operation of the program. 



The positive reaction of program parents. 

The implementation of an effective in-service program. 

The satisfactory communication between project coordinator, 
teachers, and school principals. 

The efforts of the project staff to provide a variety of instruc- 
tional experiences for the students. 
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APPENDIX A 
PROGRAM INSTRUCTION 



In order to facilitate the portrayal of ORB classroom? , a list of descriptive 
categories was formulated A scale of 0 to 3 was used to rate the frequency of 
occurrence of each teaching situation for each ORB teacher, based on four to five 
observations of her classes by the evaluators. A rating of 0 signifies that the par- 
ticular activity has never been observed; 1 describes an activity that was oc- 
casionally observed; 2 means that the activity has been observed a number of 
times; a rating of 3 indicate^ that the activity was observed during all visits to 
the class. The evaluators attempte(; to rate each teacher on each category on the 
basis of classroom observations , and interviews. Although these ratings are, by 
their nature, somewhat subjective, the evaluators feel that they present a general- 
ized picture of the classroom activities in the ORB Program. The following p'lges 
define each category and describe its occurrence. 

Individual Reading Assignments : This category describes activities where 
students are assigned passages to read in various texts. Each student may have 
a different assignment, or two to three students may be reading the same text. 
When this type of activity is occurring, the teacher usually will move from student 
to student or group to group, and give assistance or ask questions to assess pro- 
gress. 

More than one-half of tlie teachers assign individual reading activities 
for a moderate or extensive amount of the time. Nevertheless, a considerable 
number of teachers rarely or never give individual reading assignments. These 
younger students , especially the first graders , do not function well on their own, 
expecially in a reading situation. Thus , they are seldom given individual read- 
ing assignments . 

Reading Text as a Class : This category differs from "Individual Read- 
ing Assignments" in that all students are reading the same material. This 
activity is a class activity , and the teacher is conducting a group lesson. Each 
student takes a turn reading aloud while the other students follow in their texts , 
Generally, when this type of instruction occurs, the teacher monitors the read- 
ing and corrects the students^ errors where necessary. Often this activity is 
follow^ed by £. question and answer period, either using questions in the text 
or teacher prepared questions about the reading selection. 

The teachers seldom use this technique; it is completely inconsistent with 
the objectives of the program. In cases where it is used extensively, the ORB 
program coordinator has consulted the teachers involved so that they can revise 
th?ir methods and implement more effective instruction. 



Flashcards or Word Drills : In this type of activity the teacher holds up 
flashcards before the class , and students identify the stimulus on them . The 
term "word drills" refers to the t^acher^s writing of words on the chalkboard, 
followed by students reading them. The words which serve as stimuli are fre- 
quently selected to meet specifically diagnosed needs; in this way, these exer- 
cises can be tools for individualizing instruction. Observations indicate, how- 
ever, that this is better suited to be a class exercise rather than an individual 
one. The vast majority of teachers seldom use this exercise. 

Worksheets : Duri^t; this type of activity students complete worksheets 
from a published series , or prepared by the teacher. Use of worksheets can be 
tailored to individual student needs. More than 80% of the teachers make some 
use of wcrkshjets. 

Games : The use of games occurs in almost all program classrooms. Some 
teachers used games regularly , in almost every class. Other teachers seldom use 
liiem, or reserve them for late afternoon or Friday classes. Some games have been 
specifi'^ally designed for classroom use. Others are popular gsrnies, which help 
the students practice word usage. 

The use of educational games and toys is consistent with the program objec- 
tives dealing with student attitudes. In nany cases, students enrolled in tne ORB 
Program have an intense dislike for school and reading, as a result of years of 
failure and fi'ustration . The use of games and toys with £in educational theme 
encourages these students to participate in exercises which they enjoy and from 
which they can learn. However, these games do have limited usefulness. Un- 
fortunately , it appears to the evaluators that , in many situations , games are used 
solely for their entertainment value; this use might be acceptable on isolated 
occasions , such as just before a holiday, or f<.r a reward. It appears that there 
is room for some further study of the optimally effective use and timing of games. 
Such a game as Parchesi, observed in one cr two rooms , appears to have little 
educational value in a school reading program, and should be discouraged, in the 
absence of compelling evidence of its relevant value. 

Use of Audio-Visual Equipment : The ORB Program does not have a large 
quantity of equipment available for use by its teachers. All teachers do have 
cassette re'^orders , supplied by ORB , and some have access to such equipment 
as controlled readers, language masters, or phonographs. 

Almost half of the teachei^s observed have not used any of this equipment. 
Those who do use equipment either have borrowed it r?om the schools in which 
they teach, or have brought in personal equipment. Students enjoy using these 
devices . Although equipment is considerably more expensive than games , it 
is valuable in maintaining student interest, as well as effective for instruction. 
The possibility of obtaining and distributing more equipment to the teachers 
in the ORB Program for next year should be explored. 
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Variety of Activities During Period : Teachers frequently changed the 
lesson at some time during the period to prevent the students from becoming 
bored. This rating describes the extent to which mc»e than one activity was 
conducted during an ORB period. 

Forty-five minutes can be a considerable length of time, especially for 
younger students. Moreover, the conditions under which many of the ORB 
classes are held are very poor. In addition, many of the students in the ORB Pro- 
gram are difficult to motivate because of their backgroun ^s; it is not difficult 
therefore to see the importance of var>ang the students' activities throughout 
the reading period. Only a very small percentage of the teachers conducted a 
single activity each period. By far the majority of teachers plan at least tw^ types 
of activities for each class, which number, in most cases, was adequate. One 
activity will be a work activity, the second activity will be a high interest acti- 
vity . This sequence was observed to give students incentive to complete the 
work activity, so they could then participate in an activity which they liked bet- 
ter. 

Only one teacher observed by the evalustors varied her activities exten- 
sively . This teacher^s classroom has several different interest areas around the 
room, and students move from one activity to another as time permits. The for- 
mat seems to maintain interest, although tl e Ic sons were judged to be individ- 
ualized . 

Innovative Techniques : This category was used to rate the variety and 
creativity" of instructional activities. As noted earlier in these pages, students 
in the ORB Program are generally difficult to motivate , and are not particularly 
good workers. Several teachers have devised ways to stimulate their students' 
interest in reading, so that they will work harder. Almost all teachers are using 
behavior modification techniques to some extent; for this discussion, it is not 
considered an innovative technique. 

In rating a teacher in this category the evaluators tried to note those 
activities which teachers used to motivate students, especially those techniques 
which the teacl|ers appeared to have created themselves. For example, one 
teacher had her students write down all the rules of the games of hockey and 
football, and she posted these on the board. Each day the students read the 
cha:/ts until they knew and understood all the words. The students in this class 
were quite proud of the fact that they could read the hockey rules , and could 
demonstrate theii proficiency for the evaluators . The same teacher made puppets 
at home and had her students write anc. act out a play with them for the regular 
classroom students. She also had her students grow plants in little pots and 
used this experiment to teach her students the names of flowers. This teacher 
also used a slide projector, a tape unit, and a record player, in her classes. 

Needless to say , this teacher was the outstanding example of those 
teachers in this category. Most teachers were judged to use innovative techni- 
Q ques infrequently, if ever. 
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Success Experiences : Many of the children in the ORB Program hav^ 
been below grade level in their school work for some time; therefo^^e , they are 
very discouraged and have a low self-image. One objective of the ORB Program 
is to provide these students with specific success experiences to improve their 
self-image and self-confidence. The presence of success experiences in these 
classes requires designing each student's program so that he can do his work 
successfully , and then giving praise to that student for completing the work 
satisfactorily. The overwhelming majority of the teachers appear to be aware 
of this goal and attempt to meet it with ^heir students. 

Individualization : One major goal of the program is individualizing in- 
struction, so that each student can receive instruction in his weak areas and 
thus have his particular needs met. Teachers were encouraged to limit their 
class sizes to four to five students so that this individualization could occur. 
Unfortunately , the teachers have met with little success in meeting this objec- 
tive. One problem, may be class size; few have kept their classes to four to 
five students. A second reaso^^ given by many was that they did not have time 
to prepare individualized lessons for 30 students. Others indicated that the con- 
ditions under which they had to teach prevented this type of instruction; in some 
cases this appeared a valid reason . Seventeen percent of the teachers never 
appeared to be individualizing their instruction; that is, all of their activities 
were oriented toward the group as a whole. It may be that these teachers were 
fortunate enough to have classes which are homogeneous in ability. Indeed, 
homogeneity of their classes is another reason often gave for not individualizing 
lessons. But, in addition to individualized instruction, individualized attention 
is also a goal of the ORB Program. 

Forty-one percent of the teachers used some individualized instruction. 
Thirty-one percent used a moderate amount, and ten percent of the teachers 
used individualized instruction extensively. These figures challenge the state- 
ment that there is insufficient time to individualize instruction for thirty students. 
What may be needed is instruction in this teaching technique . 

Availability of Classrooms : This category indicates whether the ORB 
class is held in a regular classroom or the teachers were required to use other 
facilities , such as hallways , cloakrooms , or libraries. A rating of 0 means that 
no classroom was available; a rating of 3 means that the teacher has a classroom. 
One of the teachers has a classroom available for two of her afternoon groups. 
A 1 and 2 indicates that part-time use of a classroom was available to the 
teache . 

t appears, as a general assessment, that teachers with classrooms have 
received higher ratings, which could be expected. The evaluators have discus- 
sed this problem extensively, and review here the recommendation that every 
effort be made to improve the teaching locations in the majority of the ORB 
classes . 
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As mentioned in previous reports , the problems in getting the ORB Pro- 
gram organized for the 1971-71. academic year were considerable. One of the 
pressing problems has been lack of materials. The coordinator has done a good 
job in ordering and distributing materials to the teachers. They now report 
that they have sufficient books, texts, supplies, and equipment to conduct their 
classes. 

The one complaint concerning materials was from ORB teachers in the 
upper level classes — grades 4,5, and 6. These teachers complained that 
they could not locate the proper level of reading materials. Most of the ORB 
materials were too easy for these children, and the proper materials for that 
level has not yet been received . 
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Table A 



Comparison of 1971-1972 and 1970-1971 Final Grades for 
Program Students — Grade 1 



Subject 


Positive 
Change 


Negative 
Change 


No 
Change 


t 


P 


Reading 


17 


3 


12 


2.89 


< .01 


Arithmetic 


16 


2 


8 


2.89 


< .01 


Language 


4 


0 


1 


1.50 


NS 


Conduct 


3 


1 


1 


0.50 


NS 


Effort 


4 


2 


n 


0.42 


NS 






Table B 








Comparison of 1971-1972 and 1970- 
Program Students — 


1971 Final 
Grade 2 


Grades 


for 




Subject 


Positive 
Change 


Negative 
Change 


No 
Change 


t 


P 


Reading 


81 


26 


30 


5.22 


< .001 


Arithmetic 


62 


30 


34 


3.23 


< .01 


Language 


54 


34 


38 


2.03 


< .05 


Spelling 


7 


4 


1 


0.57 


NS 


Social Studies 


2 


0 


2 


0.71 


NS 


Conduct 


40 


43 


25 


0.22 


NS 


Effort 


51 


44 


23 


0.61 


NS 



55 

Table C 

Comparison of 1971-1972 and 1970-1971 Final Grades for 
Program Students — Grade 3 



Subject 


Positive 
Change 


Negative 
Change 


No 
Change 




t 




P 


Reading 


65 


26 


43 


3, 


,98 


< 


.001 


Arithmetic 


55 


50 


34 


0, 


,39 




NS 


Language 


57 


32 


50 


2, 


,54 


< 


.05 


Spelling 


58 


33 


34 


2, 


,52 


< 


.05 


Social Studies 


8 


0 


12 


2, 


,50 


< 


.05 


Conduct 


42 


32 


56 


1, 


,05 




NS 


Effort 


52 


37 


40 


1, 


,48 




NS 



Table D 

Comparison of 1971-1972 and 1970-1971 Final Grades for 
Program Students — Grade 4 





Subject 


Positive 
Change 


Negative 
Change 


No 
Change 




t 


P 




Reading 


70 


39 


18 


2, 


,87 


< .01 




Arithmetic 


36 


58 


28 


2, 


,17 


< .05 




Language 


49 


42 


30 


0, 


,63 


NS 




Spelling 


45 


49 


28 


0 


.31 


NS 




Social Studies 


34 


44 


20 


1, 


,02 


NS 




Conduct 


38 


37 


35 


0, 


,00 


NS 


ERIC 


Effort 


41 


38 


30 


0, 


,23 


NS 



Table E 



Comparison of 1971-1972 and 1970-1971 Final Grades for 
Program Students - Grade 5 





o u uj eu I 




INcgclLiVtr 








P 




Change 


Change 


Change 








Reading 


48 


39 


31 


0, 


,86 


NS 


Arithmetic 


42 


64 


22 


2 


.04 


< .05 


Language 


59 


21 


42 


4, 


,14 


< .001 


Spelling 


53 


34 


31 


1 


.93 


NS 


Social Studies 


55 


30 


30 


2 


.60 


< .01 


Conduct 


32 


32 


30 


-0, 


.13 


NS 


Effort 


39 


26 


28 


1 


.49 


NS 



Table F 

Comparison of 1971-1972 and 1970-1971 Final Grades for 
Program Students — Grade 6 



Subject 


Positive 
Change 


Negative 
Change 


No 
Change 




t 


P 


Reading 


47 


31 


16 


1 


.70 


NS 


Arithmetic 


42 


29 


24 


1, 


.42 


NS 


Language 


39 


39 


19 


-0. 


.11 


NS 


Spelling 


5.9 


29 


22 


1 


.09 


NS 


Social Studies 


41 


31 


21 


1 


.06 


NS 


Conduct 


34 


22 


25 


1 


.47 


NS 


Effort 


32 


21 


27 


1 


.37 


NS 



Table G 

Behavioral Objectives - Grade 1 
Sign Test 



Objective # 


Positive Change 

% 


No Change 

% 


IN egaiive L/nange 

0 
0 


z 


1 




Oil 


a 
D 


U . / i IN o 


0 


D 


on 


c 
D 


- U . DU INo 


Q 


1 A 


/y 


c 
D 


i . U / * 


A 




7 9 


Q 

o 




D 




bo 


Q 




e 
D 


99 


lb 




/ . UU INo 


7 
1 




07 

y / 




U . UU INo 


0 


oU 


CO 

b^ 


o 
o 


1 O n :k 




lb 




1 1 


n o 1 XT o 
0 . ol Mo 


10 


6 


85 


9 


0.00 NS 


11 


20 


63 


17 


0.00 NS 


12 


31 


55 


14 


1.25 NS 


13 


44 


39 


17 


1.91* 


14 


44 


52 


4 


2.78* 


15 


47 


45 


8 


2.89* 


16 


50 


28 


22 


1.76* 


17 


42 


52 


6 


2.93* 



^ NS - Not Significant * p <.05 

ERIC 



Table G (Cont.) 



Behavioral Objectives - Grade 1 
Sign Test 



Objective # 


Positive Change 

0 

•8 


No Change 

0 

•8 


Negative Change 

0. 
•5 


z 


18 


38 


31 


31 


0.40 NS 


19 


31 


47 


22 


0.45 NS 


20 


33 


53 


14 


1.46 NS 


21 


29 


50 


21 


0.49 NS 


22 


30 


50 


15 


1.46 NS 


23 


35 


50 


21 


0.91 NS 


24 


44 


44 


12 


2.27* 


25 


53 


35 


12 


2.77* 



NS - Not Significant * p <.05 



59 



Table H 

Behavioral Objectives - Grade 2 
Sign Test 



Objective # 


Positive Change 
% 


No Change 
% 


Negative Change 
% 


z 


1 


A 1 


47 


12 


2.69* 


2 




•i D 


8 


3. 58* 


3 




0 1 


13 


3. 16* 


4 


0 1 


U 4 


15 


1.43 NS 


5 


37 


DU 


1 'I 

10 


9 1 fi* 


6 


>i 0 


/I Q 
4 0 


4 


4 . 23* 


7 




D U 


Q 


0 00* 


8 


44 


4 4 


1 9 


9 Qfi* 


9 


Ztf 


0 4 


17 




10 


4D 


4 Z 


1 '\ 

1.0 


2 55* 


11 


4D 


4*^ 
4 0 


12 


2 . 96* 


12 


00 


41 
4 1 


26 


0 55 NS 


1,0 


29 


38 


33 


0.18 NS 


14 


31 


47 


22 


0.59 NS 


15 


42 


44 


14 


2.45* 


16 


37 


41 


22 


1.27 NS 


17 


42 


41 


17 


2.14* 


18 


46 


31 


23 


1.83* 


19 


37 


42 


21 


1.27 NS 


20 


41 


50 


9 


2.94* 



NS - Not Significant * p < .05 
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Table I 

Behavioral Objectives - Grade 3 
Sign Test 



Objective # 


Positive Change 


No Change 


Negative Change 


Z 


% 


% 


0 

o 





ERIC 



NS - Not Significant * p<.05 



1 31 46 23 0.57 NS 

2 52 35 13 3.28* 

3 42 52 6 3.60* 

4 49 35 16 2.63* 

5 42 34 14 3.16* 

6 51 31 18 2.71* 

7 46 44 10 3.21* 

8 51 37 12 2.94* 

9 35 26 39 0.16 NS 

10 33 40 27 0.37 NS 

11 44 23 33 0.79 NS 

12 33 30 37 0.17 NS 

13 42 37 21 1.75* 

14 42 39 19 1.85* 

15 39 33 28 0.86 NS 

16 37 37 26 0.70 NS 

17 32 44 24 0.57 NS 



Table I (Cont.) 



Behavioral Objectives - Grade 3 
Sign Test 



Objective # 


Positive Change 

G 
0 


No Change 

% 


Negative Change 

% 


z 


18 




40 


21 


1 43 NS 


19 


43 


37 


20 


1.93* 


20 


44 


33 


23 


1.69* 


21 


23 


71 


6 


2,05* 


22 


23 


63 


14 


0.90 NS 


23 


23 


69 


8 


1.75 NS 


24 


17 


71 


12 


0.57 NS 


25 


17 


71 


12 


0.57 NS 



NS - Not Significant * p <.05 
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